A great deal of effort has recently been focused on developing electronic circuits which emulate higher-order brain functions such as memory learning and/or perception/recognition.
One class of circuit devices which sense an input event and output a pattern of signals which identifies that event are associated networks. Association networks generally take the form of a matrix comprised of horizontal lines crossing and contacting an equally sized set of vertical lines. The horizontal lines simulate the functions of axons in the cortex of the brain and are used as inputs. The vertical lines simulate the function of dendrites extending from neurons. Each vertical line terminates at a summing device which acts to simulate the function of the neuron cell body. An example of such an associative network is found in co-pending applicaiton entitled "Semiconductor Cell For Neural Network Employing A Four-Quadrant Multiplier", Ser. No. 283,553, filed Dec. 9, 1988, which is assigned to the assignee of the present application.
Within an associative network, neural synapses are simulated by circuit cells which provide electrical connection between the horizontal and vertical lines of the network. Individual synapses provide a weighted electrical connection between an input and a summing element, i.e., a neuron body. A neuron consists of a neuron body, the synapse which modulates its inputs, and axons which distribute its output.
These synapse cells may be either analog or digital in nature. For an analog implementation, the weighted sum of input signals is usually computed by summing analog currents or charge packets. For a general description of an associative network processing unit consisting of analog connection elements, see "VLSI for Artificial Intelligence", edited by Jose G. DelGado-Frias and Will R. Moore, Kluwer Academic Publishers, 1989, pp. 230-233.
One of the most difficult and critical tasks faced by researchers in the field of neural networks is the integration of the synapse cells, also referred to as contact structures. The several realizations that have been proposed range from non-progammable binary to programmable analog interconnections.
In an analog synapse cell, considerations of cell size and resolution of the connection weight must be carefully balanced. Furthermore, learning within an associative network requires adaptive weight values since a typical network system cycles through a series of weight changes until the entire network converges to a certain pattern which depends upon the pattern of inputs applied. Several synapse cell circuits are described in co-pending applications "EXCLUSIVE-OR Cell For Neural Network Cell And The Like", Ser. No. 309,247, filed Feb. 10, 1989; and "EXCLUSIVE-OR Cell For Pattern Matching And Employing Floating Gate Devices", Ser. No. 325,380, filed Mar. 17, 1989, both of which are assigned to the assignee of the present application.
FIG. 2 of U.S. Pat. No. 4,802,103 of Faggin et al., discloses a contact structure which utilizes a floating gate transistor 34. Device 34 is used to discharge a target line of the network in proportion to the amount of charge stored on the floating gate member of device 34. The magnitude of the convergence response of the network is altered by incrementally erasing the floating gate transistors. In other words, the connection strength is increased to increase the discharge current associated with the target line. A detecting circuit indicates a convergence response once a predetermined amount of charge is removed from the target line.
The chief drawback of the contact structure of Faggin's FIG. 2 is that it operates as a simple one-quadrant device. That is, Faggin's synapse cell only produces a positive activation function, corresponding to an activated excitatory connection. It is well understood that biological memories accommodate both excitatory and inhibitory connections--thus providing both positive and negative responses. A cell providing both excitatory and inhibitory connections would more closely resemble the actual function performed by a synapse within the human brain. Moreover, such a cell would have the potential to learn quicker, thereby providing faster convergence within an associative network. What is needed therefore is an integrated multi-quadrant synapse cell which can produce both positive and negative responses.
As will be seen, the present invention covers a synapse cell employing one or more floating gate transistors. Various embodiments of the invention offer the advantage of multi-quadrant performance which provides the ability to make inhibitory as well as excitatory connections within an associative network. Furthermore, the cell of the present invention achieves very high densities while still providing full incremental learning capabilities.
Other prior art known to Applicant includes: U.S. Pat. No. 4,760,437 of Denker et al.; U.S. Pat. No. 4,660,166 of Hopfield; U.S. Pat. No. 4,782,460 of Spencer, "Programmable Analog Synapses For Micro Electronic Neural Networks Using A Hybrid Digital-Analog Approach", by F. J. Mack et al., IEEE International Conference on Neural Networks, Jul. 24-27, San Diego, Calif.; and "A Pipelined Associative Memory Implemented in VLSI", by Clark et al., IEEE Journal of Solid-State Circuits, Vol. 24, No. 1, pp. 28-34, Feb. 1989.